Imaging apparatus, imaging method, and computer readable storage medium

ABSTRACT

An imaging apparatus includes an imaging unit that generates a pair of pieces of image data mutually having a parallax by capturing a subject, an image processing unit that performs special effect processing, which is capable of producing a visual effect by combining a plurality of pieces of image processing, on a pair of images corresponding to the pair of pieces of image data, and a region setting unit that sets a region where the image processing unit performs the special effect processing on the pair of images.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based upon and claims the benefit of priority fromJapanese Patent Application No. 2010-286548 filed on Dec. 22, 2010 andJapanese Patent Application No. 2010-293996 filed on Dec. 28, 2010, theentire contents of which are incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an imaging apparatus that generateselectronic image data by capturing a subject, an imaging method, and acomputer readable storage medium.

2. Description of the Related Art

In recent years, technology to respond to various desires of users byproducing special effects on image data, in which a subject is captured,in an imaging apparatus such as a digital camera and a digital camcorderhas been known. For example, a technology to produce special effectssuch as a wipe and a sepia tone on image data, in which a subject iscaptured, and to store the image data in memory is known (see JapaneseLaid-open Patent Publication No. 2005-347886). In this technology, theuser can be allowed to easily select a desired special effect by causinga portion of images displayed in a display monitor to display an imageon which a special effect selected when the user operates an effectswitch to select the special effect is produced.

Also, a technology to perform mutually different image processing on amain subject such as a person and a flower and the background containedin a captured image is known (see Japanese Laid-open Patent PublicationNo. 2002-044516). In this technology, the main subject can be made tostand out from the background by performing mutually different imageprocessing, for example, sensitivity (ISO) processing, sharpnessprocessing, blurring processing, and color phase conversion processing.

However, only image processing of the same type with differentparameters is performed on the main subject and the background in aconventional imaging apparatus. Thus, properties of the main subject arenot reflected in such image processing.

SUMMARY OF THE INVENTION

An imaging apparatus according to an aspect of the present inventionincludes: an imaging unit that generates a pair of pieces of image datamutually having a parallax by capturing a subject; an image processingunit that performs special effect processing, which is capable ofproducing a visual effect by combining a plurality of pieces of imageprocessing, on a pair of images corresponding to the pair of pieces ofimage data; and a region setting unit that sets a region where the imageprocessing unit performs the special effect processing on the pair ofimages.

An imaging method executed by an imaging apparatus according to anotheraspect of the present invention includes: generating a pair of pieces ofimage data mutually having a parallax by capturing a subject; performingspecial effect processing, which is capable of producing a visual effectby combining a plurality of pieces of image processing, on a pair ofimages corresponding to the pair of pieces of image data; and setting aregion where the special effect processing is performed on the pair ofimages.

A non-transitory computer-readable storage medium according to stillanother aspect of the present invention has an executable program storedthereon, wherein the program instructs a processor to perform:generating a pair of pieces of image data mutually having a parallax bycapturing a subject; performing special effect processing, which iscapable of producing a visual effect by combining a plurality of piecesof image processing, on a pair of images corresponding to the pair ofpieces of image data; and setting a region where the special effectprocessing is performed on the pair of images.

The above and other features, advantages and technical and industrialsignificance of this invention will be better understood by reading thefollowing detailed description of presently preferred embodiments of theinvention, when considered in connection with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating the configuration of an imagingapparatus according to a first embodiment of the present invention;

FIG. 2 is a perspective view illustrating the configuration of a sidefacing a subject of the imaging apparatus according to the firstembodiment of the present invention;

FIG. 3 is a perspective view illustrating the configuration of a sidefacing a photographer of the imaging apparatus according to the firstembodiment of the present invention;

FIG. 4 is a schematic diagram illustrating an outline configuration of adisplay unit of the imaging apparatus according to the first embodimentof the present invention;

FIG. 5 is a diagram illustrating an example of a special effectprocessing information table stored in a special effect processinginformation storage unit of the imaging apparatus according to the firstembodiment of the present invention;

FIG. 6 is a schematic diagram illustrating a situation when an imagingunit of the imaging apparatus according to the first embodiment of thepresent invention generates two pieces of image data having overlappingedge portions in a left and right direction of respective fields ofview;

FIG. 7 is a diagram illustrating an example of two images correspondingto the two pieces of image data generated by a stereoscopic imagegeneration unit of the imaging apparatus in the situation illustrated inFIG. 6;

FIG. 8 is a diagram illustrating an example of an image obtained byvirtually superimposing a left-eye image and a right-eye image generatedby the stereoscopic image generation unit of the imaging apparatus inthe situation illustrated in FIG. 6;

FIG. 9 is a diagram illustrating a protrusion distance of a 3D imagevirtually recognized by a user when the left-eye image and the right-eyeimage generated by the stereoscopic image generation unit of the imagingapparatus are displayed in a display unit in the situation illustratedin FIG. 6;

FIG. 10 is a diagram illustrating an example of a screen transition of aspecial effect shooting operation menu in special effect shooting modedisplayed by the display unit when the imaging apparatus according tothe first embodiment of the present invention is set to the specialeffect shooting mode;

FIG. 11 is a flow chart illustrating an overview of processing performedby the imaging apparatus according to the first embodiment of thepresent invention;

FIG. 12 is a diagram illustrating an example of the image displayed bythe display unit of the imaging apparatus according to the firstembodiment of the present invention;

FIG. 13 is a diagram illustrating an overview of processing performed bya position difference calculation unit of the imaging apparatusaccording to the first embodiment of the present invention;

FIG. 14 is a diagram illustrating an overview of processing performed bythe position difference calculation unit of an imaging apparatusaccording to a first modification of the first embodiment of the presentinvention;

FIG. 15 is a diagram illustrating a change method of changing athreshold set by a threshold setting unit in accordance with anoperation signal input from a touch panel by a threshold change unit ofan imaging apparatus according to a second modification of the firstembodiment of the present invention;

FIG. 16 is a diagram illustrating an example of the special effectprocessing information table stored in the special effect processinginformation storage unit of an imaging apparatus according to a thirdmodification of the first embodiment of the present invention;

FIG. 17 is a schematic diagram illustrating the situation when animaging apparatus according to a fourth modification of the firstembodiment of the present invention generates two pieces of image datahaving overlapping edge portions in the left and right direction ofrespective fields of view;

FIG. 18 is a block diagram illustrating the configuration of an imagingapparatus according to a second embodiment of the present invention;

FIG. 19 is a diagram illustrating an example of a first special effectprocessing information table stored in a first special effect processinginformation storage unit of the imaging apparatus according to thesecond embodiment of the present invention;

FIG. 20 is a diagram illustrating an example of a second special effectprocessing information table stored in a second special effectprocessing information storage unit of the imaging apparatus accordingto the second embodiment of the present invention;

FIG. 21 is a flow chart illustrating an overview of processing performedby the imaging apparatus according to the second embodiment of thepresent invention;

FIG. 22 is a flow chart illustrating an overview of special effectprocessing settings illustrated in FIG. 21;

FIG. 23 is a diagram illustrating an example of the image displayed bythe display unit of the imaging apparatus according to the secondembodiment of the present invention;

FIG. 24 is a flow chart illustrating an overview of special effectsetting processing performed by the imaging apparatus according to thesecond embodiment of the present invention;

FIG. 25 is a diagram illustrating an example of the image displayed bythe display unit of an imaging apparatus according to a third embodimentof the present invention;

FIG. 26 is a diagram illustrating the situation when a subject is shotby an imaging apparatus according to a first modification of the thirdembodiment of the present invention; and

FIG. 27 is a diagram illustrating an example of the screen transitiondisplayed by the display unit of the imaging apparatus in the situationillustrated in FIG. 26.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Modes to carry out the present invention (hereinafter, referred to asthe “embodiments”) will be described below with reference to drawings.However, the present invention is not limited by the embodimentsdescribed below.

First Embodiment

FIG. 1 is a block diagram illustrating the configuration of an imagingapparatus according to a first embodiment of the present invention. FIG.2 is a perspective view illustrating the configuration of a side facinga subject (front side) of the imaging apparatus according to the firstembodiment of the present invention.

FIG. 3 is a perspective view illustrating the configuration of a sidefacing a photographer (rear side) of the imaging apparatus according tothe first embodiment of the present invention. An imaging apparatus 1illustrated in FIGS. 1 to 3 is a digital single-lens reflex camera andincludes a main body unit 2 and a lens unit 3 attachable to andremovable from the main body unit 2.

As illustrated in FIGS. 1 to 3, the main body unit 2 includes a shutter10, an image sensor 11, an A/D converter 12, memory 13, an imageprocessing unit 14, an image sensor driving unit 15, a shutter drivingunit 16, an exposure processing unit 17, an AF processing unit 18, atimer 19, a main body communication unit 20, an operation input unit 21,a flash firing unit 22, a flash charging unit 23, a flash driving unit24, a display unit 25, a touch panel 26, a nonvolatile memory 27, anexternal memory 28, an external communication unit 29, a power supplyunit 30, a power unit 31, and a control unit 32.

The shutter 10 sets the state of the image sensor 11 to an exposed stateor a light blocked state. The shutter driving unit 16 is configured byusing a stepping motor or the like and drives the shutter 10 inaccordance with a release signal.

The image sensor 11 is configured by using a CCD (Charge Coupled Device)or CMOS (Complementary Metal Oxide Semiconductor) that receives andconverts light condensed by the lens unit 3 into an electric signal. Theimage sensor driving unit 15 causes the image sensor 11 to output imagedata (analog signal) to the A/D converter 12 in predetermined timing.

The A/D converter 12 generates digital image data by making an A/Dconversion of an analog signal input from the image sensor 11 andoutputs the image data to the memory 13.

The memory 13 is configured by using SDRAM (Synchronous Dynamic RandomAccess Memory). The memory 13 temporarily records image data input fromthe A/D converter 12 or information being processed in the imagingapparatus 1. More specifically, the memory 13 temporarily records imagedata sequentially output by the image sensor 11 in each frame.

The image processing unit 14 acquires image data recorded in the memory13 and performs various kinds of image processing on the acquired imagedata before outputting the image data to the memory 13. Morespecifically, the image processing unit 14 performs normal imageprocessing containing at least edge enhancement, white balance, colorcorrections, and γ corrections on image data. The image processing unit14 includes a special effect processing unit 141.

The special effect processing unit 141 performs special effectprocessing that causes a visual effect by combining a plurality ofpieces of image processing on image data. As the special effectprocessing, for example, processing combining tone curve processing,blurring processing, and image synthesis processing is performed.

The exposure processing unit 17 performs automatic exposure of theimaging apparatus 1 by deciding conditions for still image shooting ordynamic image shooting, for example, the F number setting, shutter speedor the like based on image data recorded in the memory 13 via thecontrol unit 32.

The AF processing unit 18 adjusts an auto focus of the imaging apparatus1 based on image data recorded in the memory 13 via the control unit 32.For example, the AF processing unit 18 drives the lens unit 3 so thatsharpness of a subject image to be captured becomes maximum based on thecontrast of image data.

The timer 19 clocks the time. The timer 19 has a determination functionof the shooting date/time. The timer 19 outputs date/time data to thecontrol unit 32 to add the date/time data to shot image data.

The main body communication unit 20 is a communication interface toperform communication with the lens unit 3 mounted on the main body unit2. Incidentally, the main body communication unit 20 may allow powersupplied from the power unit 31 provided in the main body unit 2 to thelens unit 3.

The operation input unit 21 includes a power switch 211 that switchesthe state of power of the imaging apparatus 1 between the on state andthe off state, a release switch 212 that inputs a release signalinstructing shooting, a shooting mode changeover switch 213 that inputsa switching signal instructing switching of various shooting modes setto the imaging apparatus 1, an operation switch 214 that inputs aninstruction signal instructing a selection or decision of varioussettings of the imaging apparatus 1, a menu switch 215 that inputs aninstruction signal instructing a display of an operation menu screen setto the imaging apparatus 1, a preview switch 216 that inputs aninstruction signal instructing a preview display of captured image data,and a dynamic image switch 217 that inputs a dynamic image releasesignal instructing dynamic image shooting. The operation switch 214includes direction buttons 214 a to 214 d of each of up and down, leftand right to make selections and settings in the menu screen and thelike and a decision button 214 e (OK button) to decide an operation bythe direction buttons 214 a to 214 d in the menu screen and the like(see FIG. 3). Incidentally, the operation switch 214 may be configuredby using a dial switch or the like.

The flash firing unit 22 is configured by using a xenon lamp or LED(Light Emitting Diode). The flash firing unit 22 fires a flash of lightas a fill flash toward a region of the field of view to be captured bythe imaging apparatus 1. The flash charging unit 23 charges by receivinga power supply from the power unit 31 to boost the voltage. The flashdriving unit 24 drives the flash firing unit 22 by applying the voltageboosted by the flash charging unit 23 to the flash firing unit 22.

FIG. 4 is a schematic diagram illustrating an outline configuration ofthe display unit 25. As illustrated in FIG. 4, the display unit 25includes a backlight 251, a display panel 252, and a parallax barrier253. The backlight 251 is configured by an LED (Light Emitting Diode) orthe like and shines light to display images from behind the panel. Thedisplay panel 252 is configured by a display panel such as the liquidcrystal and organic EL (Electro Luminescence). The parallax barrier 253is configured by the liquid crystal or the like and laminated on the topsurface of the display panel 252. The parallax barrier 253 has slitsprovided at intervals narrower than intervals of each image of thedisplay panel 252 and separates images corresponding to the right eyeE_(R) and the left eye E_(L). For example, a parallax barrier method isapplied as the parallax barrier 253. Instead of the parallax barrier253, a lenticular method by which a lens sheet in which a lenticularlens is laminated is laminated on the top surface of the display panel252 may be applied.

In the display unit 25 having the above configuration, whenthree-dimensional image (hereinafter, referred to as a “3D image”) datais input from the control unit 32, the display panel 252 displays aleft-eye image and a right-eye image alternately starting with theleftmost image in the horizontal direction under the control of thecontrol unit 32 and the parallax barrier 253 separates light emittedfrom each image of the display panel 252. Thus, left-eye images reachthe left eye E_(L) only and right-eye images reach the right eye E_(R)only. Accordingly, the user can view 3D images displayed by the displayunit 25 stereoscopically. When the display unit 25 switches the displaymode from 3D images to two-dimensional images (hereinafter, referred toas “2D images”), the parallax barrier 253 changes from a light blockedstate to a transparent state by the voltage applied to the parallaxbarrier 253 being changed from the on state to the off state so that oneof left-eye image data and right-eye image data is output to the displaypanel 252.

The touch panel 26 is provided on top of the display screen of thedisplay unit 25 (see FIG. 3). The touch panel 26 detects the positiontouched by the user based on information displayed in the display unit25 and accepts input of an operation signal in accordance with the touchposition. Touch panels generally include the resistance film type, theelectric capacitance type, and the optical type. In the presentembodiment, any type of touch panel is applicable. In the presentembodiment, the touch panel 26 functions as an input unit.

The nonvolatile memory 27 is configured by using flash memory. Thenonvolatile memory 27 stores various programs to cause the imagingapparatus 1 to operate, imaging programs according to the presentembodiment, and various kinds of data and parameters used duringexecution of programs. The nonvolatile memory 27 includes a shootingmode information storage unit 271 that stores information about variousshooting modes executed by the imaging apparatus 1, a special effectprocessing information storage unit 272 that stores information aboutspecial effect processing performed by the image processing unit 14, aleft-eye image data storage unit 273 that stores left-eye image dataused when the display unit 25 is caused to display 3D images, aright-eye image data storage unit 274 that stores right-eye image dataused when the display unit 25 is caused to display 3D images, and athreshold storage unit 275 that stores a threshold to isolate a subjectregion and a background region contained in each of captured left-eyeimages and right-eye images. Incidentally, the nonvolatile memory 27 maystore lens specific information in accordance with the type of the lensunit 3 that can be mounted on the main body unit 2.

A special effect processing information table stored in the specialeffect processing information storage unit 272 will be described. FIG. 5is a diagram illustrating an example of the special effect processinginformation table stored in the special effect processing informationstorage unit 272.

As illustrated in FIG. 5, a special effect processing information tableT1 has content of special effect processing that can be performed by thespecial effect processing unit 141 on image data stored in the memory 13and image processing in each piece of the special effect processingdescribed therein. If, for example, the special effect processing is“Fantastic focus” of “Art1”, “Tone curve processing”, “Blurringprocessing”, and “Synthesis processing” are described as processingcontent of the image processing. The synthesis processing is imageprocessing that generates image data by synthesizing image data obtainedafter tone curve processing being performed by the special effectprocessing unit 141 and image data obtained after blurring processingbeing performed by the special effect processing unit 141. Further, ifthe special effect processing is “Pop art” of “Art4”, “Hue/saturationprocessing”, and “Contrast processing” are described as processingcontent of the image processing.

The external memory 28 is configured by using a storage medium such as amemory card inserted from outside the imaging apparatus 1. The externalmemory 28 stores information about 3D image data and 2D image data bybeing inserted into the imaging apparatus 1 via a storage mediuminterface (not illustrated) and also outputs various kinds ofinformation stored therein to the display unit 25, the image processingunit 14, and the nonvolatile memory 27 under the control of the controlunit 32.

The external communication unit 29 has a function as a communicationinterface and transmits/receives data to/from external processingapparatuses such as servers (not illustrated) and personal computers(not illustrated) via a network (not illustrated) in both directions.The external communication unit 29 acquires various programs of theimaging apparatus 1 and data on special effect processing information bytransmitting/receiving data to/from external processing apparatuses andoutputs the acquired programs and data to the control unit 32.Incidentally, the external communication unit 29 is connected to anetwork by wire or wireless LAN (Local Area Network) or the like.

The power supply unit 30 supplies power of the power unit 31 to eachstructural element of the imaging apparatus 1. The power unit 31 isconfigured by using batteries attachable to and removable from theimaging apparatus 1. Incidentally, the power supply unit 30 may supplypower supplied from an external power supply (not illustrated) to eachstructural element of the imaging apparatus 1.

The control unit 32 is configured by using a CPU (Central ProcessingUnit) or the like. The control unit 32 controls the operation of theimaging apparatus 1 in a unified manner by reading and executingprograms from the nonvolatile memory 27 in accordance with an operationsignal or instruction signal from the operation input unit 21 andtransferring instructions or data corresponding to each unitconstituting the imaging apparatus 1. The control unit 32 includes astereoscopic image generation unit 321, a face detection unit 322, aposition difference calculation unit 323, a threshold setting unit 324,a region setting unit 325, a threshold change unit 326, and a displaycontrol unit 327.

The stereoscopic image generation unit 321 acquires image dataimage-processed by the image processing unit 14 from the memory 13 togenerate 3D image data by using the acquired image data. Morespecifically, the stereoscopic image generation unit 321 divides theimage data acquired from the memory 13 into left-eye image data andright-eye image data and cuts out the divided left-eye image data andright-eye image data in a predetermined length-to-width ratio (forexample, an aspect ratio 9:16) to generate 3D image data. Thestereoscopic image generation unit 321 causes the left-eye image datastorage unit 273 and the right-eye image data storage unit 274 to storethe left-eye image data and right-eye image data, respectively.Incidentally, the length-to-width ratio in which the left-eye image dataand right-eye image data divided by the stereoscopic image generationunit 321 may be made changeable from the operation input unit 21.

The face detection unit 322 detects the face of a person contained inone of a left-eye image corresponding to left-eye image data and aright-eye image corresponding to right-eye image data by patternmatching. For example, the face detection unit 322 detects the face of aperson contained in a left-eye image by pattern matching. Incidentally,the face detection unit 322 may also detect, in addition to the face ofa person, the face of an animal such as a cat and dog.

The position difference calculation unit 323 calculates a positiondifference of a subject contained in images corresponding to each of apair of pieces of image data. More specifically, the position differencecalculation unit 323 correlatingly compares positions of a subjectcontained in each of a left-eye image and a right-eye image to calculatea position difference of a subject in the left-eye image and theright-eye image. The position difference of a subject is a difference inposition in the left and right direction of an image contained in theleft-eye image and right-eye image when the left-eye and right-eyeimages are superimposed. For example, the position difference is adifference in position of a face image when frames of the left and rightimages are superimposed or a difference in position of the backgroundwhen frames of the left and right images are superimposed. The positiondifference calculation unit 323 may determine whether positions of aface image are different or positions of the background are differentwhen frames of the left and right images are superimposed. Further, theposition difference calculation unit 323 may determine whether there isany difference in position of the background when face images containedin the left and right images are superimposed or whether there is anydifference in position of faces in the left and right direction when theface images are compared by matching the background contained in theleft and right images.

The threshold setting unit 324 sets the threshold that isolates asubject region (short-range subject region) and a background region(long-range subject region) other than the subject region contained inthe left-eye image and right-eye image in accordance with the positiondifference calculated by the position difference calculation unit 323.For example, the threshold setting unit 324 sets the threshold thatisolates a subject region and a background region in accordance with themaximum value of the position difference calculated by the positiondifference calculation unit 323. The threshold setting unit 324 outputsinformation about the set threshold to the threshold storage unit 275.Incidentally, the threshold setting unit 324 may calculate a subjectdistance from the imaging apparatus 1 to the subject by using thewell-known triangulation based on the left-eye image and right-eye imageto set the threshold in accordance with the calculated subject distance.

The region setting unit 325 sets a region where special effectprocessing is performed by the image processing unit 14 on a pair ofimages based on a position difference calculated by the positiondifference calculation unit 323. More specifically, the region settingunit 325 sets a region where special effect processing is performed bythe special effect processing unit 141 on each of a left-eye image and aright-eye image based on a position difference calculated by theposition difference calculation unit 323. Further, the region settingunit 325 sets a region where the position difference calculated by theposition difference calculation unit 323 exceeds the threshold as aregion where the image processing unit 14 performs the special effectprocessing and sets a region where the position difference is equal toor less than the thresholds as a region where the image processing unit14 performs normal image processing.

The threshold change unit 326 changes the threshold set by the thresholdsetting unit 324. For example, the threshold change unit 326 changes thethreshold set by the threshold setting unit 324 in accordance with anoperation signal input from the operation input unit 21 or the touchpanel 26.

The display control unit 327 changes the display mode of the displayunit 25. More specifically, the display control unit 327 exercisescontrol to switch an image displayed in the display unit 25 to a 2Dimage or 3D image. When the display unit 25 displays a 2D image, thedisplay control unit 327 causes the display unit 25 to display iconsabout special effect processing information performed by the specialeffect processing unit 141.

The lens unit 3 includes a first optical system 41, a second opticalsystem 42, a lens driving unit 43, a diaphragm 44, a diaphragm 45, adiaphragm driving unit 46, a mirror 47, a mirror 48, a prism 49, a lenscommunication unit 50, a lens storage unit 51, and a lens control unit52. Incidentally, the lens unit 3 is an interchangeable lens that cancapture a 3D image by capturing a subject with a parallax by a pair ofoptical systems and forming left and right subject images in the imagesensor 11. Thus, the first optical system 41 and the second opticalsystem 42 have the same configuration and only one of the opticalsystems will be described.

The first optical system 41 is configured by using one lens or aplurality of lenses. The first optical system 41 condenses light from aregion of a predetermined field of view. Incidentally, the first opticalsystem 41 may have an optical zoom function that changes an image.

The lens driving unit 43 is configured by using a DC motor or the like.The lens driving unit 43 changes the focusing position, focal length andthe like of each of the first optical system 41 and the second opticalsystem 42 by synchronously moving lenses of the first optical system 41and the second optical system 42 on an optical axis L and an opticalaxis R, respectively.

The diaphragm 44 and the diaphragm 45 adjust the exposure by limitingthe amount of incident light condensed by the first optical system 41and the second optical system 42, respectively.

The diaphragm driving unit 46 is configured by using a stepping motor orthe like. The diaphragm driving unit 46 synchronously drives thediaphragm 44 and the diaphragm 45.

The mirror 47 and the mirror 48 reflect light condensed by the firstoptical system 41 and the second optical system 42 toward the prism 49,respectively.

The prism 49 reflects light reflected by each of the mirror 47 and themirror 48 toward the image sensor 11. Accordingly, a pair of left andright subject images condensed by the first optical system 41 and thesecond optical system 42, respectively, is formed in the image sensor11.

The lens communication unit 50 is a communication interface to performcommunication with the main body unit 2 in both directions when the lensunit 3 is mounted on the main body unit 2.

The lens storage unit 51 is configured by using flash memory. The lensstorage unit 51 stores various programs to operate the lens unit 3 andvarious kinds of data and parameters used during execution of programs.

The lens control unit 52 is configured by using a CPU (CentralProcessing Unit) or the like. The lens control unit 52 controls theoperation of the lens unit 3 in accordance with an instruction signalfrom the main body unit 2. More specifically, the lens control unit 52drives the lens driving unit 43 in accordance with an instruction signalfrom the main body unit 2 to focus the lens unit 3 and also drives thediaphragm driving unit 46 to change the F number. Incidentally, when thelens unit 3 is mounted on the main body unit 2, the lens control unit 52may send focusing position information and focal length information ofthe lens unit 3 and specific information identifying the lens unit 3 tothe main body unit 2. In the present embodiment, the configurationincluding the lens unit 3, the image sensor 11, and the A/D converter 12functions as an imaging unit (hereinafter, referred to as an “imagingunit 100”).

The situation when the imaging unit 100 generates two pieces of imagedata having overlapping edge portions in the left and right direction ofrespective fields of view in the imaging apparatus 1 having the aboveconfiguration will be described. FIG. 6 is a schematic diagramillustrating a situation when the imaging unit 100 generates two piecesof image data having overlapping edge portions in the left and rightdirection of respective fields of view. As illustrated in FIG. 6, theimaging unit 100 generates left-eye image data and right-eye image databy capturing a subject E1 (distance d₁) and a subject E2 (distance d₂)having different distances from the first optical system 41 and thesecond optical system 42 by the first optical system 41 and the secondoptical system 42 provided side by side with a distance D₁ therebetween.

Then, the stereoscopic image generation unit 321 divides image datagenerated by the image sensor 11 into left-eye image data and right-eyeimage data and generates a left-eye image W_(L1) and a right-eye imageW_(R1) by cutting out the divided left-eye image data and right-eyeimage data in a predetermined length-to-width ratio.

FIG. 7 is a diagram illustrating an example of two images correspondingto two pieces of image data generated by the stereoscopic imagegeneration unit 321 in the situation illustrated in FIG. 6. In FIG. 7,the left-eye image W_(L1) is an image generated by an imagecorresponding to left-eye image data generated by the first opticalsystem 41 being cut out by the stereoscopic image generation unit 321.Also in FIG. 7, the right-eye image W_(R1) is an image generated by animage corresponding to right-eye image data generated by the secondoptical system 42 being cut out by the stereoscopic image generationunit 321. FIG. 8 is a diagram illustrating an example of an imageobtained by virtually superimposing the left-eye image W_(L1) and theright-eye image W_(R1) generated by the stereoscopic image generationunit 321 in the situation illustrated in FIG. 6. FIG. 9 is a diagramillustrating a protrusion distance of a 3D image virtually recognized bythe user when the left-eye image W_(L1) and the right-eye image W_(R1)generated by the stereoscopic image generation unit 321 are displayed inthe display unit 25 in the situation illustrated in FIG. 6.

As illustrated in FIG. 7, the stereoscopic image generation unit 321generates the left-eye image W_(L1) and the right-eye image W_(R1) inwhich positions of the subjects E1, E2 are shifted laterally due to aparallax between the first optical system 41 and the second opticalsystem 42.

Subsequently, the display control unit 327 causes the display unit 25 tosuperimpose and display the subject E2 of the left-eye image W_(L1) andthe subject E2 of the left-eye image W_(R1) (see FIG. 8). Accordingly, astereoscopic 3D image in which the subject E1 protrudes from the displaypanel 252 by ΔZ₁ in accordance with a shift amount ΔX₁ caused by theparallax of the subject E2 can virtually be recognized (see FIG. 9).

Next, the setting method of the special effect (art filter shootingmode) in which the imaging apparatus 1 performs special effectprocessing will be described. Incidentally, the imaging apparatus 1according to the present embodiment has a plurality of shooting modessuch as the shooting scene automatic judgment mode and manual shootingmode, but the setting method of only the special effect shooting modewill be described below.

First, when the imaging apparatus 1 is activated by the power switch 211being operated by the user, the control unit 32 judges whether theshooting mode of the imaging apparatus 1 is set to the special effectshooting mode. More specifically, the control unit 32 judges whether theshooting mode changeover switch 213 is set to the special effectshooting mode (ART) (see FIGS. 2 and 3). If the shooting mode of theimaging apparatus 1 is not set to the special effect shooting mode, thecontrol unit 32 causes the display unit 25 to display an operation menuscreen or a live view image corresponding to the other shooting mode.

If the shooting mode of the imaging apparatus 1 is set to the specialeffect shooting mode, on the other hand, the display control unit 327causes the display unit 25 to display a special effect shootingoperation menu screen corresponding to the special effect shooting mode.

FIG. 10 is a diagram illustrating an example of the screen transition ofthe special effect shooting operation menu in special effect shootingmode displayed by the display unit 25 when the imaging apparatus 1 isset to the special effect shooting mode.

As illustrated in FIG. 10, if the imaging apparatus 1 is set to thespecial effect shooting mode, the display control unit 327 causes thedisplay unit 25 to display a special effect shooting operation menuscreen W11 (see FIG. 10( a)). In the special effect shooting operationmenu screen W11, information about each piece of special effectprocessing that can be performed by the imaging apparatus 1 is displayedas icons A1 to A6.

Subsequently, when the user operates an up button 214 a or a down button214 b of the operation switch 214, the display control unit 327 causesthe display unit 25 to switch content of the special effect shootingoperation menu screen W11 displayed by the display unit 25 in accordancewith an operation signal input from the operation switch 214. Morespecifically, the display control unit 327 changes the color of theicons A1 to A6 displayed in the special effect shooting operation menuscreen W11 in accordance with an operation signal input from theoperation switch 214 and causes the display unit 25 to display the titleof the special effect processing corresponding to the selected icon, forexample, “Fantastic focus”, “Rough monochrome”, or “Diorama” (FIG. 10(a)→FIG. 10( b)→FIG. 10( c)). Accordingly, the user can select thedesired special effect processing and can also recognize the selectedspecial effect processing intuitively.

Then, the control unit 32 sets the special effect processing currentlydisplayed in the special effect shooting operation menu screen W11 bythe display unit 25 when the decision button 214 e of the operationswitch 214 is operated by the user, for example, “Fantastic focus” (seeFIG. 10( a)) to the imaging apparatus 1 as the special effect processingin special effect shooting mode. Incidentally, information about the setspecial effect processing is stored in the memory 13.

Subsequently, the display control unit 327 causes the display unit 25 todisplay a live view image corresponding to image data obtained after thespecial effect processing selected in the special effect shootingoperation menu screen W11 being performed by the image processing unit14 thereon. Accordingly, the user can decide the composition of shootingwhile checking a live view image obtained by performing the selectedspecial effect processing on captured image data in real time.

Next, processing performed by the imaging apparatus 1 according to thepresent embodiment will be described. FIG. 11 is a flow chartillustrating an overview of processing performed by the imagingapparatus 1.

As illustrated in FIG. 11, the control unit 32 first judges whether theimaging apparatus 1 is in 3D special effect shooting mode (step S101).If the imaging apparatus 1 is in 3D special effect shooting mode (stepS101: Yes), the imaging apparatus 1 proceeds to step S102 describedlater. On the other hand, if the imaging apparatus 1 is not set to the3D special effect shooting mode (step S101: No), the imaging apparatus 1proceeds to playback display processing in step S124 described later.

A case when, in step S101, the imaging apparatus 1 is set to the 3Dspecial effect shooting mode (step S101: Yes) will be described. In thiscase, the display control unit 327 causes the display unit 25 to displaya live view image of a 3D image corresponding to a pair of pieces ofimage data generated successively at fixed minute intervals by theimaging unit 100 (step S102). More specifically, the display controlunit 327 causes the display unit 25 to display a live view image of a 3Dimage generated by left-eye image data and right-eye image data beingisolated by the stereoscopic image generation unit 321 from a pair ofpieces of image data generated by the imaging unit 100.

If the menu switch 215 is operated when the display unit 25 displays alive view image (step S103: Yes), the display control unit 327 switchesthe display mode of the live view image displayed by the display unit 25from the 3D image to the 2D image and causes the display unit 25 todisplay icons about special effect processing (step S104). Morespecifically, as illustrated in FIG. 12, the display control unit 327causes the display unit 25 to display a live view image W21 a, specialeffect icons A11 to A13 about special effects, and switching icons A21,A22 that receives a switching instruction of the special effect icons ona display screen W21 of the display unit 25. At this point, the displaycontrol unit 327 may cause the display unit 25 to display the live viewimage W21 a as a 3D image.

Subsequently, if a special effect icon displayed by the display unit 25is operated (step S105: Yes), the control unit 32 changes the specialeffect processing set to the imaging apparatus 1 to the special effectprocessing corresponding to the special effect icon (step S106) and theimaging apparatus 1 proceeds to step S107.

If the release switch 212 is operated and a release signal instructingshooting is input (step S107: Yes), the imaging unit 100 generatesleft-eye image data and right-eye image data and outputs the generatedleft-eye image data and right-eye image data to the memory 13 (stepS108).

Subsequently, the position difference calculation unit 323 acquires theleft-eye image data and right-eye image data from the memory 13 andcalculates a position difference of subjects contained in a left-eyeimage and a right-eye image corresponding to the acquired left-eye imagedata and right-eye image data, respectively (step S109), and thethreshold setting unit 324 sets the threshold that isolates a subjectregion and a background region contained in each of the left-eye imageand the right-eye image in accordance with the maximum value of theposition difference of subjects calculated by the position differencecalculation unit 323 (step S110).

FIG. 13 is a diagram illustrating an overview of processing performed bythe position difference calculation unit 323. In FIG. 13, a subject E11is positioned closer to the imaging apparatus 1 (short-range side) thana subject E12.

In the situation illustrated in FIG. 13, the position differencecalculation unit 323 compares positions of the subject E11 contained ineach of a left-eye image W_(L2) and a right-eye image W_(R2) tocalculate a position difference of the subject E11 in the left-eye imageW_(L2) and the right-eye image W_(R2). For example, if the distance fromthe right edge of the left-eye image W_(L2) to the center position ofthe subject E11 contained in the left-eye image W_(L2) is the distancefrom the right edge of the right-eye image W_(R2) to the center positionof the subject E11 contained in the right-eye image W_(R2) is X_(R1),and the position difference of the subject E11 contained in each of theleft-eye image W_(L2) and the right-eye image W_(R2) is ΔX₁, ΔX₁ isgiven by ΔX₁=X_(R1)−X_(L1).

Further, if the distance from the right edge of the left-eye imageW_(L2) to the center position of the subject E12 contained in theleft-eye image W_(L2) is X_(L2), the distance from the right edge of theright-eye image W_(R2) to the center position of the subject E12contained in the right-eye image W_(R2) is X_(R2), and the positiondifference of the subject E12 contained in each of the left-eye imageW_(L2) and the right-eye image W_(R2) is ΔX₂, ΔX₂ is given byΔX₂=X_(R2)−X_(L2). Because the subject E11 is positioned closer to theimaging apparatus 1 than the subject E12, the position difference ΔX₁ ofthe subject E11 is larger than the position difference ΔX₂ of thesubject E12.

Thus, the position difference calculation unit 323 compares positions ofa subject contained in each of the left-eye image W_(L2) and theright-eye image W_(R2) to correlatingly calculate a position differenceof the subject in the left-eye image W_(L2) and the right-eye imageW_(R2). Further, the threshold setting unit 324 sets the threshold thatisolates a subject region and a background region in accordance with themaximum value of the position difference calculated by the positiondifference calculation unit 323. For example, in the situationillustrated in FIG. 13, if the subject distance from the subject E11 tothe imaging apparatus 1 is more than 0.6 m, the threshold setting unit324 sets a region where the position difference of the subject is morethan 1/100 as a subject region. Thus, the threshold setting unit 324sets a value smaller than the position difference ΔX₁ as the threshold.

Returning to FIG. 11, the description of step S110 and thereafter willcontinue. In step S111, the control unit 32 judges whether a regionwhere the position difference of a subject calculated by the positiondifference calculation unit 323 exceeds the threshold is present in eachof the left-eye image W_(L2) and the right-eye image W_(R2). If a regionwhere the position difference of a subject exceeds the threshold ispresent in each of the left-eye image W_(L2) and the right-eye imageW_(R2) (step S111: Yes), the imaging apparatus 1 proceeds to step S112.

Subsequently, the region setting unit 325 sets a region where theposition difference exceeds the threshold as a region where the imageprocessing unit 14 performs the special effect processing settings andsets a region where the position difference is equal to or less than thethresholds as a region where the image processing unit 14 performsnormal image processing (step S112). For example, in the situationillustrated in FIG. 13, the region setting unit 325 sets a region of thesubject E11 contained in each of the left-eye image W_(L2) and theright-eye image W_(R2) as a region where the image processing unit 14performs the special effect processing and sets a region other than theregion of the subject E11 as a region where the image processing unit 14performs normal image processing.

Then, the control unit 32 judges whether the face detection unit 322 hasdetected any face of a person in the region to which the region settingunit 325 has set the special effect processing (step S113). If the facedetection unit 322 has detected the face of a person in the region towhich the region setting unit 325 has set the special effect processing(step S113: Yes), the imaging apparatus 1 proceeds to step S114.

Subsequently, the image processing unit 14 performs the special effectprocessing set to the imaging apparatus 1, for example, “Fantasticfocus” (see FIG. 5) on the region to which the special effect processingis set by the region setting unit 325 (step S114) and performs normalimage processing on the region to which the normal image processing isset (step S115). Accordingly, shot images have a background exploitingqualities and properties of the lens unit 3 and a main subject isvisually enhanced from the background.

Then, the control unit 32 judges whether the storage mode set to theimaging apparatus 1 is the 3D image storage mode (step S116). If thestorage mode set to the imaging apparatus 1 is the 3D image storage mode(step S116: Yes), the control unit 32 generates a 3D image file fromleft-eye image W_(L2) data and right-eye image W_(R2) data on whichspecial effect processing and/or image processing has been performed bythe image processing unit 14 (step S117). The format of a 3D image fileis the MPO (Multi Picture Object) method or 3D-AVI method.

Subsequently, the control unit 32 causes the external memory 28 to storethe generated image file data (step S118) and judges whether the imagingapparatus 1 is turned off by the power switch 211 being operated (stepS119). If the imaging apparatus 1 is not turned off (step S119: No), theimaging apparatus 1 returns to step S101. On the other hand, if theimaging apparatus 1 is turned off (step S119: Yes), the imagingapparatus 1 ends the present processing.

A case when, in step S116, the storage mode set to the imaging apparatus1 is not the 3D image storage mode (step S116: No) will be described. Inthis case, the control unit 32 generates an image file from right-eyeimage W_(R2) data on which special effect processing and/or imageprocessing has been performed by the image processing unit 14 (stepS120) and the imaging apparatus 1 proceeds to step S118. The format ofthis image file is the JPEG method or AVI method. Incidentally, thecontrol unit 32 may generate an image file from the left-eye imageW_(L2) on which special effect processing and/or image processing hasbeen performed by the image processing unit 14.

A case when the face detection unit 322 has not detected any face of aperson in the region to which the region setting unit 325 has set thespecial effect processing (step S113: No) will be described. In thiscase, the image processing unit 14 performs special effect processingthat is different from the special effect processing set to the imagingapparatus 1, for example, “Pop art” (see FIG. 5) on the region to whichthe special effect processing is set by the region setting unit 325(step S121) and the imaging apparatus 1 proceeds to step S115.

A case when, in step S111, a region where the position difference of asubject exceeds the threshold is not present in the left-eye imageW_(L2) and the right-eye image W_(R2) (step S111: No) will be described.In this case, the region setting unit 325 sets normal image processingby the image processing unit 14 to the entire region of the left-eyeimage W_(L2) and the right-eye image W_(R2) (step S122).

Subsequently, the image processing unit 14 performs the normal imageprocessing on the entire region of the left-eye image W_(L2) and theright-eye image W_(R2) (step S123) and the imaging apparatus 1 proceedsto step S116.

A case when, in step S101, the imaging apparatus 1 is not set to theshooting mode (step S101: No) will be described. In this case, theimaging apparatus 1 performs playback display processing to cause thedisplay unit 25 to display a 2D image corresponding to 2D image data ora 3D image corresponding to 3D image data stored in the external memory28 (step S124) and the imaging apparatus 1 proceeds to step S119.

According to the first embodiment of the present invention describedabove, the position difference calculation unit 323 compares positionsof a subject contained in a pair of images to calculate a positiondifference of the subject in the pair of images and the region settingunit 325 sets a region where the image processing unit 14 performsspecial effect processing on the pair of images based on the positiondifference of the subject in the pair of images calculated by theposition difference calculation unit 323. As a result, properties of amain subject can be reflected.

Further, according to the first embodiment of the present invention, asubject and the background are automatically isolated to perform specialeffect processing and normal image processing, respectively. Therefore,there is no need for the user to isolate a subject and the backgroundwhile viewing an image displayed in a display monitor using specialsoftware for image processing on a personal computer to performdifferent image processing on the subject and the background and then tosynthesize the subject and the background on which image processing hasbeen performed.

Also, in the first embodiment of the present invention, the thresholdsetting unit 324 sets the threshold that isolates a subject region and abackground region in accordance with the maximum value of a positiondifference of a subject contained in each of a left-eye image and aright-eye image calculated by the position difference calculation unit323, but the threshold may be multiplied by a variable, for example, ½.

Also, in the first embodiment of the present invention, while thethreshold setting unit 324 sets the threshold that isolates a subjectregion and a background region in accordance with the maximum value of aposition difference of a subject contained in each of a left-eye imageand a right-eye image calculated by the position difference calculationunit 323, for example, the threshold may be set in accordance with adifference of the number of pixels of the subject contained in each ofthe left-eye image and the right-eye image. Further, the threshold setin accordance with the difference of the number of pixels of the subjectmay be multiplied by a variable.

Also, in the first embodiment of the present invention, the imageprocessing unit 14 performs normal image processing on a backgroundregion, but special effect processing that is different from the specialeffect processing performed on a subject region may be performed.

Also, in the first embodiment of the present invention, the imageprocessing unit 14 performs special effect processing on a subjectregion, but, for example, normal image processing may be performed onthe subject region. In this case, the image processing unit 14 performsspecial effect processing that enhances the subject region against thebackground region, i.e., special effect processing combining monochromeimage processing and saturation/hue conversion processing.

First Modification of the First Embodiment

In the embodiment described above, the main subject is a person, but theembodiment can also be applied to a plant or an animal. FIG. 14 is adiagram illustrating an overview of processing performed by the positiondifference calculation unit 323 of the imaging apparatus 1 according tothe first modification of the first embodiment of the present invention.

In the situation illustrated in FIG. 14, the position differencecalculation unit 323 compares positions of a subject E21 contained in aleft-eye image W_(L3) and a right-eye image W_(R3) to calculate aposition difference of the subject E21 in the left-eye image W_(L3) andthe right-eye image W_(R3). For example, if the distance from the rightedge of the left-eye image W_(L3) to the center position of the subjectE21 contained in the left-eye image W_(L3) is X_(L3), the distance fromthe right edge of the right-eye image W_(R3) to the center position ofthe subject E21 contained in the right-eye image W_(R3) is X_(R3), andthe position difference of the subject E21 contained in each of theleft-eye image W_(L3) and the right-eye image W_(R3) is ΔX₃, ΔX₃ isgiven by ΔX₃=X_(R3)−X_(L3).

Next, the position difference calculation unit 323 compares positions ofa subject E22 contained in each of the left-eye image W_(L3) and theright-eye image W_(R3) to calculate a position difference of the subjectE22 contained in each of the left-eye image W_(L3) and the right-eyeimage W_(R3). If the distance from the right edge of the left-eye imageW_(L3) to the center position of the subject E22 contained in theleft-eye image W_(L3) is X_(L4), the distance from the right edge of theright-eye image W_(R3) to the center position of the subject E22contained in the right-eye image W_(R3) is X_(R4), and the positiondifference of the subject E22 contained in each of the left-eye imageW_(L3) and the right-eye image W_(R3) is ΔX₄, ΔX₄ is given byΔX₄=X_(R4)−X_(L4).

Lastly, the position difference calculation unit 323 compares positionsof a subject E23 contained in each of the left-eye image W_(L3) and theright-eye image W_(R3) to calculate a position difference of the subjectE23 contained in each of the left-eye image W_(L3) and the right-eyeimage W_(R3). If the distance from the right edge of the left-eye imageW_(L3) to the center position of the subject E23 contained in theleft-eye image W_(L3) is X_(L5), the distance from the right edge of theright-eye image W_(R3) to the center position of the subject E23contained in the right-eye image W_(R3) is X_(R5), and the positiondifference of the subject E23 contained in each of the left-eye imageW_(L3) and the right-eye image W_(R3) is ΔX₅, ΔX₅ is given byΔX₅=X_(R5)−X_(L5).

Thus, the position difference calculation unit 323 compares positions ofa subject contained in each of the left-eye image W_(L3) and theright-eye image W_(R3) to correlatingly calculate a position differenceof the subject contained in the left-eye image W_(L3) and the right-eyeimage W_(R3).

Subsequently, the threshold setting unit 324 sets a threshold C thatisolates a main subject region and a background subject region based ona position difference calculated by the position difference calculationunit 323. For example, in the situation illustrated in FIG. 14, if thesubject distance from the imaging apparatus 1 to a subject is less than0.6 m (macro shooting), the threshold setting unit 324 sets a regionwhere the position difference of the subject is more than 1/20 as asubject region. Thus, the threshold setting unit 324 sets a valuesmaller than the position difference ΔX₄ as the threshold.

Then, because the position differences of the subjects E21, E22calculated by the position difference calculation unit 323 are largerthan the threshold C (ΔX₃>ΔX₄>C), the region setting unit 325 setsspecial effect processing by the image processing unit 14 to eachsubject region of the subjects E21, E22 contained in the left-eye imageW_(L3) and the right-eye image W_(R3). In contrast, because the positiondifference of the subject E23 calculated by the position differencecalculation unit 323 is smaller than the threshold C (C>ΔX₅), the regionsetting unit 325 sets normal image processing by the image processingunit 14 to the background region of the subject E23 contained in theleft-eye image W_(L3) and the right-eye image W_(R3).

Subsequently, if the subject E21 or E22 is not judged by the facedetection unit 322 to be a person, the image processing unit 14 performsspecial effect processing, for example, “Pop art” (see FIG. 5) on thesubject region of each of the subjects E21, E22 and performs normalimage processing on the region other than the subjects E21, E22.

According to the first modification of the first embodiment of thepresent invention described above, the first embodiment can be modifiedto be applicable to a plant or an animal. Further, the region settingunit 325 can set a region of special effect processing by the imageprocessing unit 14 even if a plurality of subjects are contained in eachof a left-eye image and a right-eye image.

In the first modification of the first embodiment of the presentinvention described above, the image processing unit 14 performs thesame special effect processing on the subject region of each of thesubjects E21, E22, but different special effect processing may beperformed on the subject region of each of the subjects E21, E22.

Second Modification of the First Embodiment

In the first embodiment of the present invention described above, thethreshold setting unit 324 sets the threshold that isolates a subjectregion and a background region contained in each of a left-eye image anda right-eye image in accordance with the maximum value of a positiondifference of a subject calculated by the position differencecalculation unit 323, but the threshold can be changed in accordancewith an operation signal input from the touch panel 26. FIG. 15 is adiagram illustrating a change method of changing the threshold set bythe threshold setting unit 324 in accordance with an operation signalinput from the touch panel 26 by the threshold change unit 326 of theimaging apparatus 1 according to the second modification of the firstembodiment of the present invention.

As illustrated in FIG. 15, the user first touches a 2D image W31 adisplayed on a display screen W31 of the display unit 25 in a regionwhere a desired subject is displayed. In this case, the positiondifference calculation unit 323 calculates a position difference of thesubject E11 touched by the user in accordance with an operation signalinput from the touch panel 26 (FIG. 15( a)). At this point, thethreshold setting unit 324 sets the threshold in accordance with themaximum value of the position difference calculated by the positiondifference calculation unit 323.

Subsequently, the user touches one of the special effect icons A11 toA13 concerning special effect processing displayed by the display unit25. In the situation illustrated in FIG. 15, if the user touches thespecial effect icon A11, the image processing unit 14 references thespecial effect processing information table T1 stored in the specialeffect processing information storage unit 272 to perform the specialeffect processing in accordance with an operation signal input from thetouch panel 26, for example, “Fantastic focus” on a whole 2D image W31 b(FIG. 15( b)).

Then, the user touches the switching icon A21 or the switching icon A22that instructs the change of the threshold. In this case, the thresholdchange unit 326 changes the threshold set by the threshold setting unit324 in accordance with an operation signal input from the touch panel26. Accompanying the change, the region setting unit 325 sets specialeffect processing to a region where the position difference exceeds thethreshold and also sets normal image processing to a region where theposition difference is equal to or less than the threshold. Then, theimage processing unit 14 performs the special effect processing and thenormal image processing on the regions to which the region setting unit325 sets the special effect processing and the normal image processing,respectively (FIG. 15( c)).

According to the second modification of the first embodiment of thepresent invention described above, the threshold that isolates a subjectregion and a background region can manually be changed. As a result, theregion of special effect processing by the image processing unit 14 canbe checked by an intuitive operation while viewing a live view imagedisplayed by the display unit 25.

While the threshold change unit 326 changes the threshold in accordancewith an operation signal input from the touch panel 26 in the secondmodification of the first embodiment of the present invention describedabove, the threshold may be changed in accordance with an operationsignal, for example, input from the operation input unit 21.

Third Modification of the First Embodiment

In the first embodiment of the present invention described above, thecombination of image processing in each piece of special effectprocessing performed by the image processing unit 14 is the same, but aparameter θ of image processing combined with each piece of specialeffect processing can be changed in accordance with the positiondifference of a subject or the subject.

FIG. 16 is a diagram illustrating an example of the special effectprocessing information table stored in the special effect processinginformation storage unit 272 according to a third modification of thefirst embodiment of the present invention.

As illustrated in FIG. 16, a special effect processing information tableT2 has content of special effect processing that can be performed oneach of left-eye image data and right-eye image data by the specialeffect processing unit 141 and image processing of each piece of thespecial effect processing described therein. Further, the strength ofimage processing in each piece of special effect processing isdescribed. If, for example, the subject is “Person” and “Fantasticfocus” of special effect processing “Art1” is selected, “Tone curveprocessing”, “Blurring processing”, and “Synthesis processing” aredescribed as processing content of the image processing. Further,“(Weak)” is described for “Tone curve processing” and “(Strong)” isdescribed for “Blurring processing”.

Thus, if the position difference of a subject is large, for example,1/100 is exceeded, the image processing unit 14 can perform optimalspecial effect processing on the region of the subject contained in eachof left-eye image data and right-eye image data in accordance with theposition difference of the subject by changing the level of strength ofcombined image processing for each piece of the special effectprocessing. Further, the image processing unit 14 can perform optimalspecial effect processing in accordance with each subject.

Fourth Modification of the First Embodiment

In the first embodiment of the present invention described above, a pairof pieces of image data is generated by using the 3D shooting-capablelens unit 3, but, for example, only one optical system may be configuredto generate a pair of pieces of image data by continuously capturingimages in this configuration. More specifically, as illustrated in FIG.17, a pair of pieces of image data having a parallax with respect to asubject may be generated by capturing a subject E100 continuously whilean imaging apparatus 200 being moved (arrow C) from left to right by theuser.

Second Embodiment

Next, the second embodiment of the present invention will be described.An imaging apparatus according to the second embodiment is differentfrom an imaging apparatus according to the first embodiment in theconfiguration of only a control unit and nonvolatile memory. Thus, onlythe nonvolatile memory and the control unit of an imaging apparatusaccording to the second embodiment will be described below. The samereference numerals are attached to the same structural elements belowfor the description.

FIG. 18 is a block diagram illustrating the configuration of an imagingapparatus according to the second embodiment of the present invention.An imaging apparatus 101 is a digital single-lens reflex camera andincludes the main body unit 2 and the lens unit 3 attachable to andremovable from the main body unit 2.

Nonvolatile memory 270 is configured by using flash memory. Thenonvolatile memory 270 stores various programs to cause the imagingapparatus 101 to operate, imaging programs according to the secondembodiment, and various kinds of data and parameters used duringexecution of programs. The nonvolatile memory 270 includes a shootingmode information storage unit 271 that stores information about variousshooting modes executed by the imaging apparatus 101, a special effectprocessing information storage unit 272 that stores information aboutspecial effect processing performed by the image processing unit 14, aleft-eye image data storage unit 273 that stores left-eye image dataused when the display unit 25 is caused to display 3D images, aright-eye image data storage unit 274 that stores right-eye image dataused when the display unit 25 is caused to display 3D images, and athreshold storage unit 275 that stores a threshold to isolate ashort-range region (subject region) and a background region from theregion of each of a captured left-eye image and right-eye image.

The special effect processing information storage unit 272 includes afirst special effect processing information storage unit 272 a and asecond special effect processing information storage unit 272 b. Thefirst special effect processing information storage unit 272 a storesfirst special effect processing information that provides the degree ofrecommendation in accordance with characteristics determined by acharacteristic determination unit 1325 and the combination of specialeffect processing that can be performed on each of a left-eye image anda right-eye image by the image processing unit 14. The second specialeffect processing information storage unit 272 b stores second specialeffect processing information that provides the degree of recommendationof special effect processing that can be performed on each of a left-eyeimage and a right-eye image by the image processing unit 14.

A control unit 320 is configured by using a CPU or the like. The controlunit 320 controls the operation of the imaging apparatus 101 in aunified manner by reading and executing programs from the nonvolatilememory 270 in accordance with an operation signal or instruction signalfrom the operation input unit 21 and transferring instructions or datacorresponding to each unit constituting the imaging apparatus 101. Thecontrol unit 320 includes a stereoscopic image generation unit 321, aface detection unit 322, a parallax amount calculation unit 1323, anisolation unit 1324, the characteristic determination unit 1325, aregion setting unit 1326, and a display control unit 1327.

The parallax amount calculation unit 1323 calculates a parallax amountof a pair of images by superimposing a similar region contained in eachof the pair of images. More specifically, the parallax amountcalculation unit 1323 calculates a parallax amount of a left-eye imageand a right-eye image by superimposing a similar region, for example, abackground region contained in each of the left-eye image and theright-eye image.

The isolation unit 1324 isolates a short-range region and a backgroundregion from the region of each of a pair of images in accordance with aparallax amount calculated by the parallax amount calculation unit 1323.More specifically, the isolation unit 1324 isolates a region in each ofa left-eye image and a right-eye image where the parallax amount isequal to or more than the threshold stored in the threshold storage unit275 as a short-range region and a region where the parallax amount isless than the threshold as a background region.

The characteristic determination unit 1325 determines characteristics ofa face detected by the face detection unit 322 by using knowntechnology. More specifically, the characteristic determination unit1325 determines whether characteristics of a face detected by the facedetection unit 322 belong to a man, woman, baby (child), or animal. Whena baby is determined according to a determination method, positions offeature points of the face detected by the face detection unit 322, forexample, eyes, the nose, and lips are extracted and the extractedfeature points are compared with standard sample data of the face of ababy. Further, when an animal is determined, the characteristicdetermination unit 1325 determines whether a face detected by the facedetection unit 322 is an animal by extracting feature data of the facedetected by the face detection unit 322 by using a plurality of piecesof sample data of the face of an animal such as a dog and cat andjudging whether the feature data are equal to or more than setthresholds.

If the face detected by the face detection unit 322 is in a short-rangeregion, the region setting unit 1326 sets the background region as aregion where the image processing unit 14 performs special effectprocessing. If the characteristic determination unit 1325 determinescharacteristics of the face detected by the face detection unit 322, theregion setting unit 1326 references the first special effect processinginformation stored in the first special effect processing informationstorage unit 272 a to set the special effect processing with the highestdegree of recommendation as the processing to be performed by the imageprocessing unit 14. If the face detected by the face detection unit 322is not in a short-range region or the characteristic determination unit1325 cannot determine characteristics of the face detected by the facedetection unit 322, the region setting unit 1326 references the secondspecial effect processing information stored in the second specialeffect processing information storage unit 272 b to set the specialeffect processing with the highest degree of recommendation as theprocessing to be performed by the image processing unit 14. Further, ifthe face detected by the face detection unit 322 is not in a short-rangeregion or the characteristic determination unit 1325 cannot determinecharacteristics of the face detected by the face detection unit 322, theregion setting unit 1326 sets all regions of a left-eye image and aright-eye image as a region of special effect processing performed bythe image processing unit 14.

The display control unit 1327 changes the display mode of the displayunit 25. More specifically, the display control unit 1327 exercisescontrol to switch an image displayed in the display unit 25 to a 2Dimage or 3D image. When the display unit 25 displays a 2D image, if anoperation signal is input from the operation input unit 21, the displaycontrol unit 1327 causes the display unit 25 to display icons aboutspecial effect processing information performed by the special effectprocessing unit 141.

The situation when the imaging unit 100 generates two pieces of imagedata having overlapping edge portions in the left and right direction ofrespective fields of view in the imaging apparatus 101 having the aboveconfiguration will be described. As illustrated in FIG. 6, the imagingunit 100 generates left-eye image data and right-eye image data on theimage sensor 11 by capturing a subject E1 (distance d₁) and a subject E2(distance d₂) having different distances from the first optical system41 and the second optical system 42 by the first optical system 41 andthe second optical system 42 provided side by side with a distance (baselength) D₁ therebetween.

Subsequently, the stereoscopic image generation unit 321 divides imagedata generated by the image sensor 11 into left-eye image data andright-eye image data. Then, the stereoscopic image generation unit 321generates a left-eye image and a right-eye image by cutting out thedivided left-eye image data and right-eye image data in a predeterminedlength-to-width ratio.

As illustrated in FIG. 7, the stereoscopic image generation unit 321generates a left-eye image W_(L1) and a right-eye image W_(R1) in whichpositions of the subjects E1, E2 are shifted laterally due to a parallaxbetween the first optical system 41 and the second optical system 42.

Subsequently, the display control unit 1327 causes the display unit 25to superimpose and display the subject E2 of the left-eye image W_(L1)and the subject E2 of the right-eye image W_(R1) (see FIG. 8).Accordingly, a stereoscopic 3D image in which the subject E1 protrudesfrom the display panel 252 by ΔZ₁ in accordance with a shift amount ΔX₁caused by the parallax of the subject E1 can virtually be recognized(see FIG. 9).

Then, the parallax amount calculation unit 1323 calculates a parallaxamount by superimposing the subject E2 of the left-eye image W_(L1) andthe subject E2 of the right-eye image W_(R1). For example, in thesituation illustrated in FIG. 7, if the width of the left-eye imageW_(L1) or the right-eye image W_(R1) is X₀ the parallax amountcalculation unit 1323 calculates a parallax amount in accordance withthe ratio of the width X₀.

Subsequently, the isolation unit 1324 judges whether to isolate ashort-range region and a background region from the region of each ofthe left-eye image W_(L1) and the right-eye image W_(R1) in accordancewith a parallax amount calculated by the parallax amount calculationunit 1323. More specifically, in the situation illustrated in FIG. 8,the isolation unit 1324 isolates a region where the parallax amount isshifted by 1/100 (1%) or more with respect to the width X₀ as ashort-range region. In the situation illustrated in FIG. 6, for example,if the width X₀ of a captured image is 35 mm, a distance d₁ from thefirst optical system 41 and the second optical system 42 to the subjectE1 is 1000 mm, a distance F from the first optical system 41 and thesecond optical system 42 to the image sensor 11 is 35 mm, a distance D₁from the center of the first optical system 41 to the center of thesecond optical system 42 is 10 mm, and the parallax amount with respectto the background (infinite) is ΔX, ΔX is calculated asΔX=(F×D₁)/d₁=(35×10)/1000=0.35. Because the parallax amount ΔX is 1/100(=0.35/35) of the width X₀ under this situation, the isolation unit 1324can isolate a short-range region and a background region from each ofthe left-eye image W_(L1) and the right-eye image W_(R1). Morespecifically, the isolation unit 1324 isolates a region where theparallax amount calculated by the parallax amount calculation unit 1323is equal to or more than the threshold stored in the threshold storageunit 275 as a short-range region from each of the left-eye image W_(L1)and the right-eye image W_(R1).

Thus, the parallax amount calculation unit 1323 calculates a parallaxamount by superimposing the subject E2 of the left-eye image W_(L1) andthe subject E2 of the right-eye image W_(R1). Then, the isolation unit1324 isolates a short-range region and a background region from theregion of each of the left-eye image W_(L1) and the right-eye imageW_(R1) in accordance with the parallax amount calculated by the parallaxamount calculation unit 1323.

Next, processing content of special effect processing performed by theimage processing unit 14 in special effect shooting mode will bedescribed with reference to FIGS. 19 and 20. FIG. 19 is a diagramillustrating an example of the first special effect processinginformation table stored in the first special effect processinginformation storage unit 272 a. FIG. 20 is a diagram illustrating anexample of the second special effect processing information table storedin the second special effect processing information storage unit 272 b.

As illustrated in FIG. 19, a first special effect processing informationtable T11 has special effect processing that can be performed by theimage processing unit 14 on each of a left-eye image and a right-eyeimage in special effect shooting mode and content of image processing ofeach piece of the special effect processing described therein. Further,the first special effect processing information table T11 has the degreeof recommendation in accordance with the combination of characteristicsof a face determined by the characteristic determination unit 1325 andspecial effect processing that can be performed by the image processingunit 14 on each of a left-eye image and a right-eye image describedtherein. If, for example, the characteristic determination unit 1325determines a face in “Fantastic focus” of the special effect processing“Art1” in the set special effect shooting mode as “Man”, “Tone curveprocessing (weak)”, “Blurring processing (strong)”, and “Synthesisprocessing” are described as processing content and “4” is described asthe degree of recommendation. The synthesis processing is imageprocessing that generates image data by synthesizing image data obtainedafter tone curve processing being performed by the image processing unit14 and image data obtained after blurring processing being performed bythe image processing unit 14. If the characteristic determination unit1325 determines a face in “Fantastic focus” of the special effectprocessing “Art1” in the set special effect shooting mode as “Woman”,“Tone curve processing (strong)”, “Blurring processing (weak)”, and“Synthesis processing” are described as processing content and “1” isdescribed as the degree of recommendation.

Further, if the characteristic determination unit 1325 determines a facein “Ink wash painting” of the special effect processing “Art7” in theset special effect shooting mode as “Man”, “Color judgment blockingprocessing”, “Binarization processing”, and “Edge processing (weak)” aredescribed as processing content and “7” is described as the degree ofrecommendation. The color judgment blocking processing is imageprocessing that converts similar colors in an image to the same colorand also blocks pixels in the same size.

Thus, the first special effect processing information table T11 has thedegree of recommendation of each piece of special effect processingperformed by the image processing unit 14 and also a parameter of imageprocessing described in each piece of special effect processing changedin accordance with characteristics of a face determined by thecharacteristic determination unit 1325 described therein. If the subjectis, for example, a baby or a pet, the background of shot images tends tobe the ground and moreover, a baby or a pet is taken more lovely whenappearing plump. Thus, when the subject is a baby or a pet, the boundarybetween the subject and the background is enhanced by setting thehighest degree of recommendation for the special effect processing(Fantastic focus) that blurs the background. Accordingly, the subjectcan be made to stand out from the background in an atmosphere becomingfor an image of the subject. Though only three types of characteristicsof the subject are described in the first special effect processinginformation table T11, for example, each piece of special effectprocessing and the degree of recommendation may be described for eachage or each type of pet. Further, each piece of special effectprocessing and the degree of recommendation may be described inaccordance with the number of subjects or movement thereof.

Next, the second special effect processing information table stored inthe second special effect processing information storage unit 272 b willbe described. As illustrated in FIG. 20, a second special effectprocessing information table T12 has special effect processing that canbe performed by the image processing unit 14 on each of a left-eye imageand a right-eye image in special effect shooting mode and content ofimage processing of each piece of the special effect processingdescribed therein. When, for example, “Fantastic focus” of the specialeffect processing “Art1” in the set special effect shooting mode isselected, “Tone curve processing (weak)”, “Blurring processing(strong)”, and “Synthesis processing” are described as processingcontent and “1” is described as the degree of recommendation.

Further, when “Diorama” of the special effect processing “Art9” in theset special effect shooting mode is selected, “Hue/saturationprocessing”, “Contrast processing”, “Three-time blurring processing” and“Three-time synthesis processing” are described as processing contentand “9” is described as the degree of recommendation. The three-timeblurring processing is processing in which after the hue/saturationprocessing, the image processing unit 14 performs the blurringprocessing three times on image data on which the contrast processinghas been performed. The three-time synthesis processing is processing inwhich image data is generated by synthesizing image data before theblurring processing and image data after the blurring processing eachtime the image processing unit 14 performs the blurring processing.

Thus, the user is caused to actively select special effect processingwith a high degree of recommendation by the degree of recommendation ofspecial effect processing when the user manually sets special effectprocessing to the imaging apparatus 101, while setting the degree ofrecommendation of special effect processing assumed for 2D images low.Accordingly, the possibility of selecting special effect processing, inwhich two-dimensional positions and three-dimensional positions areincompatible when 3D images are shot, can be reduced.

Next, the processing performed by the imaging apparatus 101 according tothe second embodiment will be described. FIG. 21 is a flow chartillustrating an overview of processing performed by the imagingapparatus 101.

As illustrated in FIG. 21, the control unit 320 first judges whether theimaging apparatus 101 is in shooting mode (step S1101). If the imagingapparatus 101 is in shooting mode (step S1101: Yes), the imagingapparatus 101 proceeds to step S1102 described later. On the other hand,if the imaging apparatus 101 is not set to the shooting mode (stepS1101: No), the imaging apparatus 101 proceeds to step S1112.

A case when, in step S1101, the imaging apparatus 101 is set to theshooting mode (step S1101: Yes) will be described. In this case, thedisplay control unit 1327 causes the display unit 25 to display a liveview image of a 3D image corresponding to a pair of pieces of image datagenerated successively at fixed minute intervals by the imaging unit 100(step S1102). More specifically, the display control unit 1327 causesthe display unit 25 to display a live view image of a 3D image generatedby left-eye image data and right-eye image data being isolated by thestereoscopic image generation unit 321 from a pair of pieces of imagedata generated by the imaging unit 100.

Subsequently, the control unit 320 judges whether the shooting mode isthe special effect shooting mode (step S1103). If the shooting mode isthe special effect shooting mode (step S1103: Yes), the imagingapparatus 101 performs special effect setting processing that causes avisual effect on an image to be captured (step S1104) before proceedingto step S1105.

In contrast, if the shooting mode is not the special effect shootingmode (step S1103: No), the imaging apparatus 101 proceeds to step S1105described later.

If a release signal instructing shooting is input after the releaseswitch 212 being operated (step S1105: Yes), the imaging apparatus 101shoots (step S1106).

Subsequently, the control unit 320 creates an image file from capturedimage data (step S1107). The image file is generated from, for example,image data obtained after special effect processing being performed bythe image processing unit 14 on each of a left-eye image and a right-eyeimage generated by the imaging unit 100.

Then, the control unit 320 causes the external memory 28 to store thegenerated image file (step S1108).

Subsequently, the control unit 320 judges whether the imaging apparatus101 is turned off after the power switch 211 being operated (stepS1109). If the imaging apparatus 101 is turned off (step S1109: Yes),the imaging apparatus 101 ends the present processing.

In contrast, if the imaging apparatus 101 is not turned off (step S1109:No), the control unit 320 switches the mode of the imaging apparatus 101(step S1111) in accordance with an operation signal input from theshooting mode changeover switch 213 when the shooting mode changeoverswitch 213 is operated (step S1110: Yes) and the imaging apparatus 101returns to step S1101.

A case when, in step S1101, the imaging apparatus 101 is not in shootingmode (step S1101: No) will be described. In this case, the control unit320 judges whether the imaging apparatus 101 is in playback mode (stepS1112). If the imaging apparatus 101 is not in playback mode (stepS1112: No), the imaging apparatus 101 returns to step S1101.

In contrast, if the imaging apparatus 101 is in playback mode (stepS1112: Yes), the display control unit 1327 causes the display unit 25 todisplay a list of image files stored in the external memory 28 (stepS1113).

Subsequently, when an image file is selected from the list of imagefiles displayed by the display unit 25 (step S1114: Yes), the displaycontrol unit 1327 causes the display unit 25 to display an imagecorresponding to the image data contained in the selected image file(step S1115).

Then, the control unit 320 judges whether a termination operation of theimage playback is performed (step S1116). More specifically, the controlunit 320 judges whether the playback mode is switched to the shootingmode by the shooting mode changeover switch 213 being operated. If notermination operation of image playback is performed (step S1116: No),the imaging apparatus 101 returns to step S1113 described above. On theother hand, if a termination operation of image playback is performed(step S1116: Yes), the imaging apparatus 101 proceeds to step S1109.

If, in step S1105, no release signal is input (step S1105: No), theimaging apparatus 101 proceeds to step S1109.

If, in step S1110, the shooting mode changeover switch 213 is notoperated (step S1110: No), the imaging apparatus 101 returns to stepS1101.

If, in step S1114, no image file is selected from the list of imagefiles displayed by the display unit 25 (step S1114: No), the imagingapparatus 101 returns to step S1112.

Next, special effect setting processing in step S1104 illustrated inFIG. 21 will be described. FIG. 22 is a flow chart illustrating anoverview of special effect setting processing illustrated in FIG. 21.

As illustrated in FIG. 22, the parallax amount calculation unit 1323first superimposes a left-eye image and a right-eye image so that aperipheral region contained in each of the left-eye image and theright-eye image is fit to calculate a parallax amount of the left-eyeimage and the right-eye image (step S1202). For example, the parallaxamount calculation unit 1323 calculates a parallax amount of a left-eyeimage and a right-eye image by superimposing the background contained ineach of the left-eye image and the right-eye image.

Subsequently, the isolation unit 1324 performs isolation processing thatisolates a short-range region and a background region from the region ofeach of the left-eye image and the right-eye image in accordance withthe parallax amount calculated by the parallax amount calculation unit1323 (step S1203). More specifically, the isolation unit 1324 judgeswhether there is any region where the parallax amount calculated by theparallax amount calculation unit 1323 is equal to or more than thethreshold stored in the threshold storage unit 275. Then, if there is aregion where the parallax amount calculated by the parallax amountcalculation unit 323 is equal to or more than the threshold stored inthe threshold storage unit 275, the isolation unit 1324 isolates theregion where the parallax amount calculated by the parallax amountcalculation unit 1323 is equal to or more than the threshold stored inthe threshold storage unit 275 as a short-range region. If a short-rangeregion and a background region can be isolated from each of the left-eyeimage and the right-eye image (step S1203: Yes), the face detection unit322 detects the face of a person contained in the short-range region ofthe left-eye image or the right-eye image (step S1204).

Then, the characteristic determination unit 1325 performs characteristicdetermination processing to determine characteristics of the facedetected by the face detection unit 322 (step S1205). If characteristicsof the face detected by the face detection unit 322 can be determined(step S1205: Yes), the region setting unit 1326 references the firstspecial effect processing information table T11 stored in the specialeffect processing information storage unit 272 to set the special effectprocessing with the highest degree of recommendation in accordance withcharacteristics determined by the characteristic determination unit 1325as the processing to be performed by the image processing unit 14 ifthere is no manual setting of special effect processing by the user(step S1206). If, for example, the characteristic determination unit1325 determines that the face detected by the face detection unit 322belongs to a man, the region setting unit 1326 sets the special effectprocessing with the highest degree of recommendation (see FIG. 19) asthe processing to be performed by the image processing unit 14.

Subsequently, the image processing unit 14 performs the special effectprocessing set by the region setting unit 1326 on the background region(step S1207). Accordingly, a live view image displayed by the displayunit 25 is a 3D image in which special effect processing is performed onthe background region only so that the short-range region (main subject)is enhanced compared with the background region.

Then, if the menu switch 215 is operated (step S1208: Yes), the displaycontrol unit 1327 causes the display unit 25 to display special effecticons about special effect processing information (step S1209). Morespecifically, as illustrated in FIG. 23, the display control unit 1327causes the display unit 25 to display a live view image W121 a, specialeffect icons A11 to A14 about special effect processing, and switchingicons A21, A22 into which a switching instruction of the special effecticons is input on a display screen W121 of the display unit 25. In thiscase, the display control unit 1327 may cause the display unit 25 todisplay special effect icons of only special effect processingapplicable to 3D images.

Subsequently, if an operation to select a special effect icon displayedby the display unit 25 is performed to select the special effectprocessing (step S1210: Yes), the region setting unit 1326 changes thesetting from the special effect processing set at that time to theselected special effect processing (step S1211). Then, the imagingapparatus 101 returns to the main routine in FIG. 21.

A case when the isolation unit 1324 cannot isolate a short-range regionand a background region from each of the left-eye image and theright-eye image (step S1203: No) or the characteristic determinationunit 1325 cannot determine characteristics of the face detected by theface detection unit 322 (step S1205: No) will be described. In thiscase, the image processing unit 14 references the second special effectprocessing information table T12 stored in the second special effectprocessing information storage unit 272 b to perform the special effectprocessing with the highest degree of recommendation on the entireregion of each of the left-eye image and the right-eye image if there isno manual setting of special effect processing by the user (step S1212).

Subsequently, if the menu switch 215 is operated (step S1213: Yes), thedisplay control unit 1327 causes the display unit 25 to display specialeffect icons (see FIG. 23) about special effect processing information(step S1214).

Then, if an operation to select a special effect icon displayed by thedisplay unit 25 is performed to select the special effect processing(step S1215: Yes), the region setting unit 1326 sets the selectedspecial effect processing from the special effect processing set at thattime (step S1216). Then, the imaging apparatus 101 returns to the mainroutine in FIG. 21.

If, in step S1208, the menu switch 215 is not operated (step S1208: No),if, in step S1210, no special effect processing is selected (step S1210:No), if, in step S1213, the menu switch 215 is not operated (step S1213:No), or if, in step S1215, no special effect processing is selected(step S1215: No), the imaging apparatus 101 returns to the main routinein FIG. 21.

According to the second embodiment of the present invention describedabove, the isolation unit 1324 isolates a short-range region and abackground region from the region of each of a left-eye image and aright-eye image in accordance with a parallax amount calculated by theparallax amount calculation unit 1323 and, if the face detected by theface detection unit 322 is in the short-range region, the region settingunit 1326 sets the background region as a region where special effectprocessing is performed by the image processing unit 14. As a result,the subject can be made to stand out by the background while reality ofthe subject being maintained.

Further, according to the second embodiment of the present invention, ifthe characteristic determination unit 1325 determines characteristics ofthe face detected by the face detection unit 322, the first specialeffect processing information table T11 is referenced to set the specialeffect processing with the highest degree of recommendation as theprocessing to be performed by the image processing unit 14. Accordingly,the boundary between the subject and background can be enhanced inaccordance with characteristics of the subject and thus, the subject canbe made to stand out in an atmosphere more becoming suitable for animage of the subject.

Third Embodiment

Next, the third embodiment of the present invention will be described.The third embodiment of the present invention is different from thesecond embodiment in special effect setting processing performed by animaging apparatus. Thus, only special effect setting processingperformed by an imaging apparatus according to the third embodiment ofthe present invention will be described. An imaging apparatus accordingto the third embodiment of the present invention has the sameconfiguration as the configuration of the imaging apparatus 101according to the second embodiment described above and thus, adescription thereof is omitted.

FIG. 24 is a flow chart illustrating an overview of special effectsetting processing performed by the imaging apparatus 101 according tothe third embodiment.

As illustrated in FIG. 24, the face detection unit 322 first detects anyface contained in each of a left-eye image and a right-eye image (stepS1301).

Subsequently, the control unit 320 judges whether a face region of theface detected by the face detection unit 322 from each of the left-eyeimage and the right-eye image is equal to or more than a predeterminedvalue (step S1302). More specifically, the control unit 320 judgeswhether the face region of the face detected by the face detection unit322 from each of the left-eye image and the right-eye image occupies,for example, 5% of the entire image or more. If the face region of theface is equal to or more than the predetermined value (step S1302: Yes),the imaging apparatus 101 proceeds to step S1303.

Then, the parallax amount calculation unit 1323 superimposes the faceregion of the left-eye image and the face region of the right-eye imagedetected by the face detection unit 322 (step S1303) to calculate aparallax amount of the left-eye image and the right-eye image (stepS1304).

Subsequently, the isolation unit 1324 performs isolation processing thatisolates a short-range region and a background region from the region ofeach of the left-eye image and the right-eye image in accordance withthe parallax amount calculated by the parallax amount calculation unit1323 (step S1305). If a short-range region and a background region canbe isolated from each of the left-eye image and the right-eye image(step S1305: Yes), the imaging apparatus 101 proceeds to step S1306.

Then, the characteristic determination unit 1325 determinescharacteristics of the face detected by the face detection unit 322(step S1306). If characteristics of the face detected by the facedetection unit 322 can be determined (step S1306: Yes), the regionsetting unit 1326 references the first special effect processinginformation table T11 stored in the first special effect processinginformation storage unit 272 a to set the special effect processing withthe highest degree of recommendation in accordance with characteristicsdetermined by the characteristic determination unit 1325 as theprocessing to be performed by the image processing unit 14 if there isno manual setting of special effect processing by the user (step S1307).

Subsequently, the image processing unit 14 performs the special effectprocessing set by the region setting unit 1326 on the background regionof each of the left-eye image and the right-eye image (step S1308).Accordingly, a live view image displayed by the display unit 25 is a 3Dimage in which a special effect is applied to the background region onlyso that the short-range region is enhanced compared with the backgroundregion.

Then, if the menu switch 215 is operated (step S1309: Yes), the displaycontrol unit 1327 causes the display unit 25 to display special effecticons (step S1310).

Subsequently, if the short-range region of the 3D image displayed by thedisplay unit 25 is touched (step S1311: Yes) and an operation to operatea special effect icon is performed within a predetermined time, forexample, 3 sec. to select the special effect processing (step S1312:Yes), the image processing unit 14 performs the selected special effectprocessing on the short-range region (step S1313). Then, the imagingapparatus 101 returns to the main routine in FIG. 21.

FIG. 25 is a schematic diagram illustrating an overview of theprocessing of steps S1311 to S1313 in FIG. 24. As illustrated in FIG.25, the user first touches a short-range region E11 of a live view imageW131 a displayed on a display screen W131 by the display unit 25 (FIG.25( a)). Then, the user touches the special effect icon A11 as a desiredspecial effect icon (FIG. 25( b)). In this case, the image processingunit 14 performs the special effect processing in accordance with thespecial effect icon on the short-range region E11. Then, the usertouches the special effect icon A33 that receives input of an operationsignal instructing a decision of special effect processing by manualsettings (FIG. 25( c)). Accordingly, the image processing unit 14 canperform the special effect processing desired by the user on theshort-range region. Incidentally, the image W131 a in FIG. 25schematically illustrates a 3D image as a plane.

The description will continue by returning to FIG. 24. If, in stepS1311, the short-range region (person) of the 3D image displayed by thedisplay unit 25 is not touched (step S1311: No) and an operation tooperate a special effect icon is performed to select the special effectprocessing (step S1314: Yes), the region setting unit 1326 sets theselected special effect processing from the special effect processingset at that time for the entire image (step S1315). Then, the imagingapparatus 101 returns to the main routine in FIG. 21. If the short-rangeregion is touched, the special effect processing may be set to only thebackground region.

A case when the face region detected by the face detection unit 322 isnot equal to or more than the predetermined value (step S1302: No), ashort-range region and a background region cannot be isolated from eachof the left-eye image and the right-eye image (step S1305: No), orcharacteristics of the face detected by the face detection unit 322cannot be determined (step S1306: No) will be described. In this case,the image processing unit 14 references the second special effectprocessing information table T12 stored in the second special effectprocessing information storage unit 272 b to perform the special effectprocessing with the highest degree of recommendation on the entireregion of each of the left-eye image and the right-eye image if there isno manual setting of special effect processing by the user (step S1316).

Subsequently, if the menu switch 215 is operated (step S1317: Yes), thedisplay control unit 1327 causes the display unit 25 to display thespecial effect icons (step S1318).

Then, if an operation to select a special effect icon displayed by thedisplay unit 25 is performed to select the special effect processing(step S1319: Yes), the region setting unit 1326 sets the selectedspecial effect processing from the special effect processing set at thattime (step S1320). Then, the imaging apparatus 101 returns to the mainroutine in FIG. 21.

If, in step S1312, no special effect processing is selected (step S1312:No), the imaging apparatus 101 proceeds to step S1314.

If, in step S1314, no special effect processing is selected (step S1314:No), if, in step S1317, the menu switch 215 is not operated (step S1317:No), or if, in step S1319, no special effect processing is selected(step S1319: No), the imaging apparatus 101 returns to the main routinein FIG. 21.

According to the third embodiment of the present invention describedabove, the same effect as the effect of the second embodiment isobtained. Further, the image processing unit 14 can perform the specialeffect processing set by manual settings of the user on the short-rangeregion. As a result, different special effect processing can beperformed on each of the short-range region and the background region sothat captured images can be expressed more creatively.

In the second and third embodiments, if the characteristic determinationunit 1325 determines characteristics of the face detected by the facedetection unit 322, the first special effect processing informationtable T11 is referenced to set the special effect processing with thehighest degree of recommendation as the processing to be performed bythe image processing unit 14 and the region setting unit 1326 sets thebackground region as a region where the image processing unit 14performs special effect processing, but the special effect processingset to the background region may be changed by, for example, the menuswitch 215.

First Modification of the Third Embodiment

In the above embodiments, the region setting unit 1326 sets content ofthe special effect processing performed by the image processing unit 14in accordance with characteristics of a face determined by thecharacteristic determination unit 1325, but, for example, faces detectedby the face detection unit 322 may be registered so as to set content ofthe special effect processing to be performed by the image processingunit 14 by associating with the registered faces.

FIG. 26 is a diagram illustrating the situation when a subject is shotby an imaging apparatus 101 according to a first modification of thethird embodiment. FIG. 27 is a diagram illustrating an example of thescreen transition displayed by the display unit 25 in the situationillustrated in FIG. 26. The imaging apparatus 101 is configured in thesame manner as the imaging apparatus 101 described above and thus, adescription thereof is omitted.

As illustrated in FIGS. 26 and 27, the user decides the composition whenshooting a subject E150 while viewing, for example, a live view imageW140 a displayed on a display screen W140 in FIG. 27. In the situationillustrated in FIG. 27, the display control unit 1327 causes the liveview image W140 a to display a frame F11 corresponding to a face regiondetected by the face detection unit 322. If, at this point, the usertouches the frame F11 on the live view image W140 a displayed by thedisplay unit 25, the control unit 320 causes the nonvolatile memory 270to store face information containing characteristics of the face of theframe F11 in accordance with an operation signal input from the touchpanel 26.

Subsequently, the display control unit 1327 causes a right region R140on the display screen W140 to display special effect icons A11 to A19.Then, if the user touches one of the special effect icons A11 to A19displayed by the display unit 25 (FIG. 27( a)), the control unit 320causes the nonvolatile memory 270 to store content of the special effectprocessing corresponding to the special effect icon touched by the userby associating with the face the nonvolatile memory 270 is caused tostore.

Then, the display control unit 1327 causes the right region R140 on thedisplay screen W140 displayed by the display unit 25 to display specialeffect processing corresponding to content of the set special effectprocessing and also causes a lower region R141 of the display screenW140 to display support messages (FIG. 27( b)).

Thus, according to the first modification of the third embodiment, ifthe face detection unit 322 detects a face matching face informationstored in the nonvolatile memory 270, the region setting unit 1326 canautomatically set content of the special effect processing associatedwith the face information to the image processing unit 14.

Other Embodiments

In the above embodiments, the region setting unit may set a region of aperson including a face detected by the face detection unit as a regionwhere the image processing unit performs special effect processing.Accordingly, an image in which special effect processing is performed ona person can be generated.

In the above embodiments, if the face detection unit detects a pluralityof faces, the region setting unit may set a region of a person in acenter region taken in each of a pair of images or a person exceeding athreshold may set as a region where the image processing unit performsspecial effect processing. Accordingly, an image in which special effectprocessing is performed only on a main subject can be generated.

In the above embodiments, content of the special effect processing maybe changed in accordance with the position difference calculated by theposition difference calculation unit.

In the above embodiments, the image processing unit performs specialeffect processing during shooting, but the present processing may beperformed on a 3D image or 2D image when, for example, image data storedin the external memory is played back.

In the above embodiments, still image shooting is taken as an example,but the present processing can also be applied to dynamic imageshooting.

In the above embodiments, the imaging apparatus is assumed to be adigital single-lens reflex camera, but the present processing can alsobe applied to various electronic devices equipped with a displayfunction such as a digital camera in which a lens unit and a main bodyare integrally formed, a digital camcorder, and a mobile phone with acamera.

Additional advantages and modifications will readily occur to thoseskilled in the art. Therefore, the invention in its broader aspects isnot limited to the specific details and representative embodiments shownand described herein. Accordingly, various modifications may be madewithout departing from the spirit or scope of the general inventiveconcept as defined by the appended claims and their equivalents.

1. An imaging apparatus, comprising: an imaging unit that generates apair of pieces of image data mutually having a parallax by capturing asubject; an image processing unit that performs special effectprocessing, which is capable of producing a visual effect by combining aplurality of pieces of image processing, on a pair of imagescorresponding to the pair of pieces of image data; and a region settingunit that sets a region where the image processing unit performs thespecial effect processing on the pair of images.
 2. The imagingapparatus according to claim 1, further comprising a position differencecalculation unit that calculates a position difference of a position ofthe subject contained in the pair of images, wherein the region settingunit sets the region where the image processing unit performs thespecial effect processing on the pair of images on the basis of theposition difference calculated by the position difference calculationunit.
 3. The imaging apparatus according to claim 2, further comprisinga threshold setting unit that sets a threshold that isolates a subjectregion and a background region in each of the pair of images on thebasis of the position difference calculated by the position differencecalculation unit, wherein the region setting unit sets the region wherethe position difference calculated by the position differencecalculation unit exceeds the threshold as the region where the imageprocessing unit performs the special effect processing.
 4. The imagingapparatus according to claim 3, further comprising a face detection unitthat detects a face of a person from one of the pair of images, whereinthe image processing unit changes content of the special effectprocessing in accordance with a detection result of the face detectionunit.
 5. The imaging apparatus according to claim 4, further comprising:a display unit that displays the image corresponding to the pair ofpieces of image data; an input unit that receives input of an operationsignal; and a threshold change unit that changes the threshold set bythe threshold setting unit in accordance with the operation signal inputfrom the input unit.
 6. The imaging apparatus according to claim 5,wherein the input unit is a touch panel provided on a display screen ofthe display unit to receive the input of the operation signal inaccordance with a touch position of an object from outside.
 7. Theimaging apparatus according to claim 6, wherein the region setting unitsets the region of the person including the face detected by the facedetection unit as the region where the image processing unit performsthe special effect processing.
 8. The imaging apparatus according toclaim 7, wherein if the face detection unit detects a plurality offaces, the region setting unit sets the region of the person in a centerregion taken in each of the pair of images or the person exceeding thethreshold as the region where the image processing unit performs thespecial effect processing.
 9. The imaging apparatus according to claim8, wherein the image processing unit changes the content of the specialeffect processing in accordance with the position difference calculatedby the position difference calculation unit.
 10. The imaging apparatusaccording to claim 1, further comprising: a parallax amount calculationunit that calculates a parallax amount of the pair of images bysuperimposing a similar region contained in each of the pair of images;an isolation unit that isolates a short-range region and a backgroundregion from the region of each of the pair of images in accordance withthe parallax amount calculated by the parallax amount calculation unit;and an image processing setting unit that sets the background region asthe region where the image processing unit performs the special effectprocessing.
 11. The imaging apparatus according to claim 10, furthercomprising a face detection unit that detects a face contained in eachof the pair of images, wherein if the face detected by the facedetection unit is in the short-range region, the image processingsetting unit sets the background region as the region where the imageprocessing unit performs the special effect processing.
 12. The imagingapparatus according to claim 11, further comprising: a characteristicdetermination unit that determines characteristics of the face detectedby the face detection unit; and a first special effect processinginformation storage unit storing first special effect processinginformation that provides a degree of recommendation in accordance witha combination of the characteristics determined by the characteristicdetermination unit and the special effect processing that can beperformed by the image processing unit on the pair of images, wherein ifthe characteristic determination unit determines the characteristics ofthe face detected by the face detection unit, the image processingsetting unit references the first special effect processing informationto set the special effect processing with the high degree ofrecommendation as processing to be performed by the image processingunit.
 13. The imaging apparatus according to claim 12, furthercomprising a second special effect processing information storage unitstoring second special effect processing information that provides thedegree of recommendation of the special effect processing that can beperformed by the image processing unit on the pair of images, wherein ifthe face detected by the face detection unit is not in the short-rangeregion, the image processing setting unit references the second specialeffect processing information to set the special effect processing withthe high degree of recommendation as processing to be performed by theimage processing unit.
 14. The imaging apparatus according to claim 13,further comprising: a display unit that displays the image correspondingto the pair of pieces of image data; a display control unit that causesthe display unit to display icons about each piece of the special effectprocessing that can be performed by the image processing unit; and aninput unit that receives input of an operation signal instructing aselection of the icon displayed by the display unit, wherein the imageprocessing setting unit sets the special effect processing correspondingto the icon selected by the input unit.
 15. The imaging apparatusaccording to claim 14, wherein the region setting unit sets the regionof the person including the face detected by the face detection unit asthe region where the image processing unit performs the special effectprocessing.
 16. The imaging apparatus according to claim 12, wherein ifthe face detection unit detects a plurality of faces, the region settingunit sets the region of a center person taken in each of the pair ofimages as the region where the image processing unit performs thespecial effect processing.
 17. The imaging apparatus according to claim13, wherein the image processing unit changes content of the specialeffect processing in accordance with the parallax amount calculated bythe parallax amount calculation unit.
 18. An imaging method executed byan imaging apparatus comprising: generating a pair of pieces of imagedata mutually having a parallax by capturing a subject; performingspecial effect processing, which is capable of producing a visual effectby combining a plurality of pieces of image processing, on a pair ofimages corresponding to the pair of pieces of image data; and setting aregion where the special effect processing is performed on the pair ofimages.
 19. A non-transitory computer-readable storage medium with anexecutable program stored thereon, wherein the program instructs aprocessor to perform: generating a pair of pieces of image data mutuallyhaving a parallax by capturing a subject; performing special effectprocessing, which is capable of producing a visual effect by combining aplurality of pieces of image processing, on a pair of imagescorresponding to the pair of pieces of image data; and setting a regionwhere the special effect processing is performed on the pair of images.